Capability test method based on joint test support platform

ABSTRACT

Disclosed is a capability test method based on a joint test support platform. The method includes steps of describing an initial capability in a test, combining a capability to be developed based on the initial capability, and determining an evaluation strategy and a joint task background information of the test. Further, the method includes generating a logical shooting range for the joint test support platform according to the joint task background information, developing a test scenario according to the joint task background information and the logical shooting range, decomposing the test scenario, determining a test plan corresponding to the test scenario, executing the test according to the test plan, analyzing and evaluating a test result of the test, and generating one or more joint capability evaluation reports for the test.

TECHNICAL FIELD

The disclosure relates to the technical field of equipment/system jointtest, and more specifically, to a capability test method based on ajoint test support platform.

BACKGROUND

The equipment/system task test is a complex system engineering. It is anequipment/system test activity to assess and evaluate task efficiency,guarantee efficiency, applicability, task satisfaction and qualitystability of the equipment/system, based on the equipment/system'smission and task, under the conditions of approximate real application,in accordance with the task process. equipment/system test design refersto the overall planning and appropriate arrangement of all aspects,links and the entire process of the test work according to the testobject and purpose before the equipment/system test is implemented. Thisprocess is called test design. Test design is the planning of the entiretest process and is the core part of the task test. It determines howthe task test implementation process is carried out and is a key link indetermining the success of the test.

Therefore, the equipment/system task test design is such a process.Before the equipment/system enters the task test implementation phase,based on the task and capability requirements of the testedequipment/system, the task test purpose is scientifically determined,the test resources are comprehensively considered, the task test isscientifically planned, and the test method is scientificallydetermined, and finally a scientific and effective task test plan isformed.

The existing mature test design process is mission-driven. The so-calledmission-driven is to construct task scenarios directly based ontask/mission requirements, then to run tests using the test platform,and finally to collect and analyze the generated test data. This type oftest mode is unable to learn from previous tests performed when the testmissions is similar or inclusive. Therefore, the current design processof joint tests can easily lead to a large amount of repetitive work,resulting in the waste of resources. It can only achieve flat testdevelopment, and cannot achieve the cycle of capabilities throughout thetest process. The program design stage cannot be directly related to thecapabilities determined in the requirements definition stage.

Therefore, how to provide a device/system test design method that can bedirectly related to the capabilities of the previous device/system testis an urgent problem for those skilled in the field to solve.

SUMMARY

For this reason, the present invention provides a capability test methodbased on a joint test support platform, which can directly referenceprevious test equipment/systems in the form of capability in the testdesign phase, thereby continuously iterate in the entire test system toachieve cumulative development of equipment/system test.

In order to achieve the above objectives, the present invention adoptsthe following technical solution: a capability test method based on ajoint test support platform is provided. The capability is defined as anability to perform tasks to achieve the expected results through variousmethods and means under specific standards and conditions.

The capability test method includes the following steps.

S1: describing an initial capability in a test, and determining anevaluation strategy and a joint task background information of the testaccording to the capability to be developed based on the initialcapability.

S2: generating a logical shooting range for the joint test supportplatform according to the joint task background information.

S3: developing a test scenario according to the joint task backgroundinformation and the logical shooting range, decomposing the testscenario, and determining a test plan corresponding to the testscenario.

S4: performing the test according to the test plan; and

S5: analyzing and evaluating test results of the test, and generatingone or more joint capability evaluation reports for the test.

According to the above technical solutions, compared with the currenttechnology, by describing the capabilities of the test, mark andencapsulate the equipment/systems participating in this test. When thecapabilities to be achieved in the next test is similar to that achievedin the previous test or when the capabilities realized by previous testsare included, the previous test equipment/systems can be directly quotedin the form of capabilities during the test design stage, so as tocontinuously iterate through the entire test system to achievecumulative development of equipment/system tests. By distinguishing therequirements definition phase and the test plan design phase, the designwork in the requirement definition phase is platform-independent andcompatible with multiple platforms.

Preferably, in the above-mentioned capability test method based on ajoint test support platform, S1 includes: S11: formulating capabilityinformation and test information. The capability information isinformation describing the capability. The test information isinformation describing the test; The capability is divided into aninitial capability, a developing capability, and an output abilityaccording to the conversion method of the capability in the whole testdesign stage. The initial capability is defined as a joint capability ofthe test object relying on its own equipment/system or a knowncombination of systems or SoS at the initial stage of the test. Thedeveloping capability is defined as a joint capability formed by thetest object according to the higher-level task instructions of dividingat the test planning stage. The output capability is defined as anactual output joint capability of the test evaluation analysis stage.

S12: developing joint task background information.

Retrieving from the basic model database of the joint test environmentaccording to the capability information and test information formulatedin S11, and selecting the basic model of the joint test environmentsuitable for the test.

Selecting the equipment/system resource model and environmental resourcemodel to participate in the test according to the selected basic modelof the joint test environment.

Adding a task interaction relationship between the equipment/systemresource model and the environmental resource model participating in theexperiment to generate joint task background information.

Verifying a consistency and completeness of the generated joint taskbackground information.

S13: developing an evaluation strategy.

Collecting evaluation information.

Determining a key capability issue and key task issue according to thecapability information in S11.

Establishing an evaluation strategy framework.

The present invention realizes the cycle of capability by dividing themanifestations of capability in different test stages. By citing keycapability issues, the capabilities determined at the test plan designstage and the requirements definition stage are directly related.

Preferably, in the above-mentioned capability test method based on ajoint test support platform, the equipment/system resource modelincludes a first-level platform model and a second-level platform model;and the environmental resource model includes a physical environment anda human environment.

Preferably, in the above-mentioned capability test method based on ajoint test support platform, the evaluation information in S13 includesthe evaluation subject, the evaluation function, the evaluation area,the evaluation capability, and the evaluation capability level. Theevaluation subject is defined as the equipment, personnel orinstitutions for collecting the results of test performance andquantitatively and qualitatively evaluate the results of testperformance in the evaluation process. The evaluation function isdefined as the behaviors and actions that the evaluation subject mustperform during the evaluation process. The evaluation area is defined asthe task test area to be evaluated which the evaluation subject definesbased on the evaluation function; the evaluation capability is definedas the related capability to be evaluated. The capability level isdefined as the capability evaluation standard for evaluating anddividing the joint capability to be evaluated.

Preferably, in the above-mentioned capability test method based on thejoint test support platform, the key capability issue in S13 isanalytical statement for evaluating the performance of the joint missionrelated capability. The basic elements of the key capability issueinclude task set, mission expected effect, test standard, testcondition, test method, and test means. The key task issue is defined asthe analytical statement of design task efficiency and missionadaptability;

The evaluation strategy framework is divided into independent factorsand dependent factors according to the measured dependence. Theindependent factor includes joint missions and task, environmentalconditions, and system configuration options. The dependent factorincludes joint mission effectiveness measurement, task performancemeasurement and system property measurement.

Preferably, in the above-mentioned capability test method based on ajoint test support platform, S2 includes:

S21: instantiating a test resource. Selecting the subordinate componentsof test resource in the basic model of the joint test environment toinstantiate them.

S22: adding a test node. Determining the physical node running by thecomponent according to the instantiated component, so that the testresource participates in the test at the correct physical node.

S23: generating an intermediate plug-in. Generated a dynamic linklibrary is to realize data transmission among test resources accordingto the basic model of the joint test environment corresponding to alltest resources in the logical shooting range.

Preferably, in the above-mentioned capability test method based on thejoint test support platform, S3 includes:

S31: developing a test scenario. The test scenario is a briefdescription of the action/interaction relationship between our troops,enemy troops and the environment during the test.

S32: developing a test concept. The developing a test concept includesestablishing an overall test goal, establishing a test purpose, anddetermining a test sequence.

S33: developing a test design according to the test concept and the testscenario. The test design is a specific plan during the test operation.

Preferably, in the above-mentioned capability test method based on thejoint test support platform, S32 includes:

S321: establishing an overall test goal. Obtaining the joint task goalof this test according to the joint task background information in S12.Associating the available joint task goal with the available keycapability issues in S13. Integrating stakeholders, capability and jointmission objectives into an overall test goal.

S322: developing a test purpose. Decomposing the global test goal instep S321 into test purpose according to the key capability issues inS13, and each test purpose is based on a specific key capability issue.

S323: determining the test sequence. Sorting the developed test purposesaccording to priority and time.

Preferably, in the above-mentioned capability test method based on ajoint test support platform, S33 includes:

S331: developing test fragments.

Decomposing the test scenario specified in step S31 into a plurality oftest fragments as a test event during test operation by taking the testpurposes in S32 as a reference.

S332: configuring the subscription release relationship.

Adding a subscription release relationship for the task interactionrelationship among test resources, and determining the data transmissionrelationship and type among test resources.

S333: formulating a data collection plan.

Determining the key capability indicators that the current test fragmentneeds to collect according to the test fragments in S331 and thecorresponding key capability issues, and formulating corresponding datacollection plans.

S334: designing test samples.

Developing a corresponding test sample is formulated for each testfragment according to the key capability indicators and the datacollection plan in S331.

Preferably, in the above-mentioned capability test method based on thejoint test support platform, S334 includes:

S3341: selecting test objects, and setting test attributes andparameters.

S3342: select test factors from each test object.

S3343: setting the number of test levels and level values for eachparticipating test factor according to the key capability indicator anddata collection plan in S331.

S3344: selecting a test design method suitable for the selected testfactors.

S3345: generating test samples according to the test design method.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to explain the embodiments of the present invention or thetechnical solutions in the prior art more clearly, the following willbriefly introduce the drawings that need to be used in the descriptionof the embodiments or the prior art. Obviously, the drawings in thefollowing description are only embodiments of the present invention.

For those of ordinary skill in the art, other drawings can be obtainedbased on the provided drawings without creative work.

FIG. 1 is a flowchart provided by the present invention;

FIG. 2 is a flowchart of step S1 provided by the present invention;

FIG. 3 is an information flow diagram of step S1 provided by the presentinvention;

FIG. 4 is an example diagram of the key capability issue provided by thepresent invention;

FIG. 5 is a flowchart of step S2 provided by the present invention;

FIG. 6 is an information flow diagram of step S2 provided by the presentinvention;

FIG. 7 is a flowchart of step S3 provided by the present invention;

FIG. 8 is an information flow diagram of step S3 provided by the presentinvention;

FIG. 9 is a diagram of the relationship of test goal, test purpose, andkey capability issues provided by the present invention;

FIG. 10 is a diagram of the relationship of test scenarios, testfragments, test goal, and test purpose provided by the presentinvention.

DETAILED DESCRIPTION OF EMBODIMENTS

The technical solutions in the embodiments of the present invention willbe clearly and completely described below in conjunction with theaccompanying drawings in the embodiments of the present invention.Obviously, the described embodiments are only a part of the embodimentsof the present invention, rather than all the embodiments. Based on theembodiments of the present invention, all other embodiments obtained bythose of ordinary skill in the art without creative work shall fallwithin the protection scope of the present invention.

Referring to FIG. 1, the embodiment of the present invention discloses acapability test method based on a joint test support platform.Capability refers to the ability to perform tasks to achieve theexpected results through various methods and means under specificstandards and conditions; It is characterized by the following steps:S1: describing the initial capability in this test, and determining theevaluation strategy and the background information of the joint task ofthe test according to the capability to be developed based on theinitial capability.

S2: generating a logical shooting range for the joint test supportplatform according to the joint task background information;

S3: developing a test scenario according to the background informationof the joint task and the logical shooting range, decompose the testscenario, and determining a test plan corresponding to the testscenario;

S4, performing the test according to the test plan; and

S5: analyzing and evaluating the test results of the test, andgenerating one or more joint capability assessment reports for the test.

Among them, S1 produces two key products: the description of the JointOperational Context for Test (JOC-T) of evaluation strategy and test.JOC-T describes the operating environment in which the system orcapability is about to perform its mission. The evaluation strategyestablishes a method to verify whether the system or capability actuallymeets its development requirements.

S2 converts the JOC-T generated in step S1 into a logical shootingrange, and determines the resource model in JOC-T as a component thatcan directly participate in the test. This step will generate a logicalshooting range to configure components and object models for subsequenttest execution.

S3 designs the test based on the products generated in the steps S1 andS2, formulates the test scenario, concept and fragment, and determinesthe specific test plan.

S4 is the execution stage of the test. The test execution platform loadsand runs the test plan formulated in S3 to complete the test.S5 analyzesand evaluates the test data generated after the test is executed, andfinally generates one or more joint capability evaluation reports.

The steps S1-S3 of the present invention will be described in detailbelow.

(1) As shown in FIG. 2-3, step S extracts capability/test informationfrom the initial capability document to formulate capability/testdescriptions, and uses the formulated capability/test descriptions toassist in the development of JOC-T and evaluation strategies for use insubsequent steps.

Step S1 includes:

Step S11: formulating capability information and test information.

For the first step of test design, describing the capability to conductthe test, explaining the various attribute parameters of the capabilityto conduct the test, and formulate the capability/test description toextract the capability/test information related to the test from therelevant system files. The test designer selects/combines theinformation, and completes the description of each attribute parameterof the capability for use in subsequent steps.

Capability information is information describing capabilities, includingcapability keywords, capability text descriptions, capability parameterlists, and system attribute lists. Among them, the capability keyword isa brief description of the joint ability; the capability textdescription is the necessary explanation and supplementary descriptionof the capability; the capability parameter list is a collection ofmultiple parameters related to the capability; in general, theparameters should be numerical values with the corresponding unit. Thesystem attribute list refers to a collection of multiple attributes ofweapon equipment, systems, or SoS related to the capability.

The capability information is stored in the capability file, and thecapability is divided into the initial capability, the developingcapability, and the output capability according to the test design stageof the capability. Initial capability refers to the joint capability ofthe test object relying on its own equipment/system or the knowncombination of systems or SoS at the initial stage of the test, and itis stored in the Initial Capability Document (ICD); the developingcapability refers to the joint capability which is formed by the testobject dividing according to the higher-level task instructions at thetest planning stage, and it is stored in the Capability DevelopmentDocument (CDD); the output capacity refers to the joint capacityactually derived during the evaluation and analysis stage, and is storedin the Capability Production Document (CPD). It should be pointed outthat after the evaluation is completed and the output capability isderived, it can be used as an initial capability to participate in thenext test design process based on capability.

A list of initial capability description information composed ofmultiple capability description information is stored in the initialcapability document (ICD). The test designer loads the initialcapability by importing the initial capability document, then combinesthe initial capability and selects the key capability indicators thatthis test cares about, and imports the combined capability informationinto the capability development document (CDD) for subsequent steps.After the entire test is over, the developed capability information willbe written into the capability production document (CPD).

Test information is the information describing the test, including testname, test mission statement, test expected results, and test tasks andgoals. The test information can be obtained from relevant system files,or it can be added manually by the test designer at this stage.

Step S12: developing the joint task background information.

Realize the capability described in S11, so that the capability cantruly participate in the test. The carriers of the capability are allkinds of equipment, which have a variety of capabilities by themselvesor in combination. Include the equipment as a trial participant in thetest, that is, include the capability into the test. During the test,the operating background of the equipment is the joint task background,which integrates information such as our army, enemy forces, theenvironment, and the interaction relationship among units, and is usedfor scenario description of the joint task environment. S12 uses thecapability information formulated from capability/test description inStep S1 l, and complete the establishment of the joint task backgroundthrough the basic model of the joint test environment (includingequipment resource model and environmental resource model). The processincludes the following 5 steps:

S121: analyzing the test mission.

The first step in formulating the joint task background of the test isto analyze the test mission. The analysis process is to retrieve thebasic model of the joint test environment (including the equipmentresource model and the environmental resource model) based on thecapability/test information formulated in S11. The basic model of thejoint test environment suitable for this test is selected from the basicmodel database of test environment, which is used by test designers andlays the foundation for describing test participants and environment.

To analyze and test missions, semantic simplification must be carriedout, and extract corresponding text description information from thepreviously formulated information.

This information includes: mission description, test goals, and expectedeffects of the mission. Secondly, according to the capabilitydevelopment description information, retrieve the relevant weaponequipment model with the joint capability, and recommend it to the testdesigner, so that the test designer can quickly select and build asatisfactory joint task background based on the recommended results.

S122: describing test participants.

Participants in the test are the equipment that participates in thetest. They are the carriers of capability and the manifestation ofcapability in the joint task background.

There can be multiple test participants in the joint task background ofthe test, and the test participants directly promote the test.

In the previous step, the test designer obtained the recommendedresource model and test description information related to this test.Therefore, the test designer can use the test description information asa reference at this time, select the appropriate equipment resourcemodel from the recommended resource model as a participant in this testand add it to the test.

S123: describing the test environment.

Describing the operating environment of the equipment resource model. Asin step 2) describing the test participants, when describing the testenvironment, the test designer needs to select a suitable environmentalresource model from the recommended resource models as the operatingenvironment of the equipment resource model, and add it to the test.

The environment includes air, water, land, plants, animals and otherliving things, man-made structures, historical and cultural resources,and their interrelationships. The environment can be classified into twocategories: physical environment and humanistic environment. Thephysical environment includes natural environment and man-madeenvironment, which can be extended to the sky, weather, ocean and space.The physical environment can include external and internal conditions(for example: temperature, humidity, radiation, electric and magneticfields, shock and vibration, etc.). The humanistic environment includeslocal customs, economic conditions, ethnic conditions, political andreligious sects, the history and relationship of various social groups,and the attitudes of ordinary people towards our and enemy forces.

S124: describing the task interaction.

After adding all resource models, the relationship of the models shouldbe described.

Assuming that there are our own and enemy forces in the joint taskbackground of the test, then the relationship between our testparticipants and the enemy test participants is a hostile relationship,and the relationship of ourselves test participants or enemy testparticipants is collaborative relationship. These hostile or cooperativerelationships are called task interaction relationships. The taskinteraction relationship can be divided into two types: attack-defensetask relationship and active-passive cooperation relationship. In thetask interaction relationship, test participants will pass correspondingdata to each other to complete the interaction. Test designer can alsouse text annotations to convey information such as the specialcapability of test participants, key issues, the realtime andreliability of task interactions, and the synchronization of the jointtask background.

S125: Check JOC-T.

After completing steps 1)-4), the joint task background of the test iscompleted. In order to ensure the consistency and completeness of thejoint task background, the joint task background needs to be verified.

Verification mainly includes the consistency and completeness of jointtask. The verification of consistency refers to whether the testparticipants in the completed joint task background cover thecapability/test information described in step S11. The verification ofcompleteness refers to whether the completed joint task background hasall the information that it should have. The basic composition of thejoint task background information includes: mission descriptioninformation, description information of our side and the enemy,environment description information, and interaction information. Thispart mainly verifies whether the test designer has selected thecorresponding weapon equipment and environmental resource model based onthe test mission and capability development description information,checks whether the parameters/attributes allocated by the resource modelare correct, and verifies whether the content of the data type of thetask interaction relationship is nonvoid.

Step S13: developing an evaluation strategy.

The evaluation strategy is a blueprint/master plan for evaluating theefficiency of a joint mission of a certain capability. It explains theguidelines that the test evaluator needs to follow in the later stagesof the test. The evaluation strategy is formulated during therequirement definition stage of the test. Then, in the subsequent stagesof test design and evaluation, the test designer will refer to the needsof the test evaluator to improve the test design plan. When the testevaluator formulates the data collection plan and analysis plan, theymust abide by the previously formulated evaluation strategy. The keyelements of the evaluation strategy include Critical Capability Issue(CCI), evaluation independent factors and evaluation dependent factors.The key element refers to the element that the test designer needs topay special attention when formulating the test evaluation strategy.Evaluation of independent factors refers to factors that do not affecteach other and are independently evaluated during the evaluationprocess, including: joint missions and tasks, enemy and environmentalconditions, and system configuration options. Evaluation dependentfactors refer to the factors that are affected by independent factors inthe evaluation process, mainly including: mission measures ofeffectiveness (MMOE), task measures of performance (TMOP), and Measuresof System/SoS attributes (MOSA).

In order to complete the formulation of the evaluation strategy, thefollowing three steps are required.

S131: collecting evaluation information.

The first thing in developing an evaluation strategy is to collectevaluation information, and focus on solving the problems of “who willevaluate” and “evaluation standards”.

There are mainly evaluation subject, evaluation function, evaluationarea, and joint capability analysis standards.

The evaluation information mainly includes evaluation subjectinformation, evaluation function information, evaluation areainformation, capability information to be evaluated, and capabilitylevel information. The evaluation subject refers to the equipment,personnel or organization that collects the test execution results andevaluates quantitatively and qualitatively during the evaluationprocess. The evaluation subject information can be a certain testexecution or evaluation personnel, it can also be a test measurementequipment or system, or a certain test execution and analysisorganization. Evaluation function information is the behavior that theevaluation subject must perform in the evaluation process, and theactions performed. The evaluation area information is the test area forthe task to be evaluated that is delineated by the evaluation body basedon the evaluation function. There may be overlaps in different testevaluation areas. The capability to be evaluated describes the relatedcapability to be evaluated. The capability level information refers tothe capability evaluation standard to evaluate and classify the jointcapability to be evaluated.

S132: developing a key capability issue and a key task issue.

The most important thing in developing an evaluation strategy is toformulate critical capability issue (CCI) and critical operational issue(COI). CCI is the evaluation object and solves the “what to evaluate”problem. COI is the problems caused by the capability gap and need to bepaid attention to in the evaluation.

The key capability issue is an analytical statement that evaluates theperformance of the joint mission related competence. Its basic elementsinclude: task set, mission expected effect, test standards andconditions, test methods and means. It is generally described as:evaluating the capability to achieve the expected effect of D through Bmethods and means under A standards and conditions, as shown in FIG. 4,where the standards and conditions are the configuration information ofJOC-T, and the methods and means are the task action information in theJOC-T information. The task set is the test mission in the testinformation, and the expected effect is the test effect in the testinformation. According to the corresponding information of the above CCIbasic elements, the corresponding information is extracted from theprevious steps to complete the formulation of CCI.

The key task issue is analytical expressions involving taskeffectiveness and task adaptability. They are more used to describe therisks faced by a mission test system, that is, problems that requireadditional attention/avoidance when developing new joint capability.Defining the key mission issue needs to rely on capability gap (CG). Theso-called capability gap refers to the shortcomings between thedeveloping joint capability and the initial capability. Theseshortcomings may be due to the absence of such capabilities, and theindicators of the existing initial joint capabilities do not meet therequirements, and the existing initial joint capability needs torecombine, etc. In the process of defining key task issue, in additionto referring to the joint task background of the test, the content inthe initial capability description information and the capabilitydevelopment description information should also be cited.

S133: establishing an evaluation strategy framework.

Establishing an evaluation strategy framework is the core issue ofestablishing an evaluation strategy. It summarizes the evaluationmethodology and supporting test strategies centered on the jointmission, including the basic tasks and system capability that contributeto the effectiveness, applicability and survivability of the task. Themain purpose of this framework is to provide reference materials thatcan be quickly consulted for subsequent test design and test evaluationand analysis. Through the framework of the table, you can know theresponse variables that a task/capability needs to pay attention to whenevaluating and measuring, you can also know the overview of the testdesign, and have a forward-looking understanding of the resources andtest cycles used in the test.

The establishment of an evaluation strategy framework focuses on solvingthe problem of “evaluation methods”, that is, what basic methods areused to measure and collect data for each evaluation object. Theevaluation strategy framework does not target any specific measurementmethod. It only stipulates that for a certain evaluation object, a datameasurement and collection plan need to be formulated in accordance withthis framework.

The test evaluation strategy framework can be divided into independentfactors and dependent factors according to the measured dependence.Among them, the independent factors evaluated mainly include: jointmission and task, environmental conditions, and system configurationoptions. The dependent factors mainly include: joint missioneffectiveness measurement, task performance measurement, and systemattribute measurement.

The evaluation strategy framework can determine the goal of the test,the quantitative measurement centered on the joint mission, the keypoints of the test design, the test resources, and the test period. Thegoal of the test focuses more on the task mission and/or capabilitybeing evaluated, and links each mission or capability with at least onemission-centric quantitative measurement to identify some available keycapability issues or key task issues. Quantitative measurement centeredon the joint mission points out more of the response variables that needto be quantitatively measured in the evaluation process, and it isimpossible to express detailed and specific measurement methods orprocedures. Related measurement methods or procedures can be toimplemented until the test design stage. Some of these responsevariables are key capability parameters, and some are key systemattributes. The quantitative measurement centered on the joint missionmainly involves related response variables such as task effectiveness,task fitness and survival. The key points of test design mainly involveinformation such as the name of the test design, test points andexamples. These test design points are mainly to provide follow-up testdesigners with relevant guidance for designing test plan. Test resourcesbriefly restate and evaluate the resources or weapon equipment setneeded by the mission or joint capability. The test period brieflydescribes the evaluation period of the mission or joint capability.

(2) FIG. 5 illustrates the specific process of converting JOC-T into alogical shooting range. S2 reads the JOC-T information formulated in S1,instantiates test participants in JOC-T, and determines the specificequipment component participating the test, and add corresponding testnodes for these instantiated test participants, and generate middlewareplug-ins based on the object model. FIG. 6 illustrates the informationflow of S2.

Logical shooting range is a logical shooting range that combinesresources distributed in different regions, different services, or realor virtual or structured. It is a collection of test training resourcesthat cross regions, across virtual and real boundaries. The logicalshooting range is a collection of all weapon equipment-level models thatsupport test operations. These test resources realize thedeterminization of resources in the logical shooting range, and worktogether to complete specific test tasks through a series of sharedspecific object models, and realize the interconnection andinteroperability of independent test resources. To complete theconversion from JOC-T to logic shooting range, the following 3 steps arerequired:

S21: instantiating test resources.

In this step, the resources of the joint test environment basic modelcited in JOC-T need to be instantiated and turn into components that candirectly participate in the test. After the test resource instantiationstarts, the test designer selects its subordinate components for eachtest resource in JOC-T and instantiates it. After all the resources areinstantiated, the definition of the resources in the logical shootingrange is completed. When instantiating, not all the components of theresource model have been modeled, and the components that have not beenmodeled will be given to the modeler as an additional modelingrequirement, and the modeler will perform modeling according to therequirement.

S22: adding a test node.

The characteristics of the logical shooting range determine that allresources in the shooting range are not necessarily in the same physicalshooting range. In order to complete the joint test, the resources inthe logical shooting range are usually provided by multiple physicalshooting ranges. In order to distinguish the resources in differentphysical shooting ranges when running the test, after instantiating thetest resources, it is necessary to determine the physical node of itsoperation according to the specific components, that is, the physicalshooting range to which it belongs. So during the subsequent operationthe test, it can be ensured that these test resources will participatein the test at the correct node to ensure the reliability of the testoperation.

In order to avoid omissions, at the same time of completing theinstantiation of test resource, that is, the component, the test nodemust be determined and assigned to the corresponding test node.

S23: generating an intermediate plug-in.

According to the basic model of the joint test environment correspondingto all test resources in the logical shooting range, a dynamic linklibrary is generated to realize data transmission between testresources.

(3) FIG. 7 illustrates the specific process of S3 to generate the testplan based on the test scenario. S3 uses the JOC-T formulated in S1 andthe logical shooting range formulated in S2 to construct the testscenario, thereby determining all the test participant information intest plan; S3 uses the evaluation strategy developed in S1 to determinethe concept of the test, and provides guidance for decomposing the testscene into test fragments; S3 conducts a test plan design for each testfragment after decomposing the test scene into test fragments. FIG. 8illustrates the information flow of S3.

S31: developing a test scenario;

The test scenario is a brief description of the course of action duringthe test. It describes how our troops, enemy troops and the environmentacted/interacted during the test event. The main purpose of the taskaction in the test scenario is to allow the collection of data thatmeets the needs of test analysis.

When formulating test scenario, the joint task background and logicalshooting range of the completed test should be referred. The joint taskbackground provides information about the relationship of testparticipants, and the logical shooting range provides specific testparticipant information, such as instantiated name of resources in theshooting range, component type, belonging node and IP address, etc.

S32: developing a test concept.

The activities carried out to formulate the test concept include theestablishment of overall test goal, the establishment of test purposeand the determination of the test sequence. The completed test conceptwill provide guidance for the subsequent specific design process.

Capability is the ability to complete a set of tasks and achieveexpected goals or effects through a series of means or methods undergiven standards and conditions. The capability evaluation of weaponequipment system is a comprehensive evaluation of the equipment systemfrom the perspective of capability. Capability is the core of the entiretest design process, and test plans are divided according to thecapability to be identified. In order to clarify which capabilitiesshould be designed for the test, and how the completed test plan shouldbe implemented, an overall plan is required before the test designbegins. The test concept is the overall plan to guide the entire test.The test concept consists of two parts, the test purpose and the testsequence. The test purpose describes which capabilities should bedesigned for the test, and the test sequence describes how the designedtest plans should be carried out. The specific formulation process ofthe test concept includes the following 3 steps: S321: establishing anoverall test target.

The overall test goal focuses the test on the specific subset of jointcapabilities and the contribution to the realization of the jointmission of the system/SoS, and clarifies and refines the system/systemcapability from the perspective of joint test.

The first step to establish an overall test goal is to review the JOC-Tformulated in step S12, which records the joint task goals of this testand the work that contributes to the success of the joint task;secondly, the test designer needs to review CCI developed in step S13and link the related joint task goals with these applicable CCIs;finally, the joint task goals of stakeholders, capabilities, and relatedsystems/setups are integrated into an overall test goal. The testdesigner should check the ICD and CDD to ensure that the test objectivescan be traced back to documented capabilities and key capabilityindicators.

S322: establishing a test purpose.

The test purpose describes the direct purpose of the test and providesmore information to describe the joint task capability and itscontribution to achieving the expected results. The test purpose furtherconcentrates the overall test goal on specific CCI, and providesguidance for the subsequent development of test fragments.

In order to establish the test purpose, the first is to review the testgoal completed in the previous step, and the test purpose will bedecomposed from the test goal; the second is to review the CCIformulated in S13. The CCI will be used as a reference for thedecomposition of the test goal. A completed test purpose should be basedon a specific CCI. The relationship of test goal, test purpose and CCIis shown in FIG. 9.

Due to cost constraints such as time, capital, etc., it is impossible todesign and evaluate all test purposes. Therefore, it is necessary tomake choices and select the part that is really necessary for the test.The selected test purposes form the test purpose list which is used as areference for the test designer when formulating the test fragment.

S323: determining the test sequence.

After the test purpose list is established, it is determined whichcapabilities should be designed for the test, and the number and themesof the test events are also determined. Next, it is necessary todetermine that each test event should be performed at what time, thatis, determine the test sequence.

For a complex joint test project, there will be a large number of keycapability issues that need to be evaluated, and a large number of testevents will be generated at the same time. During the joint test,multiple test events can be carried out at the same time. This requiresa test event execution plan to determine the time when the test eventsshould be performed. The test sequence will be the overall plan duringthe test execution. The main content of the test sequence is the timeplan for the execution of the test event. The test designer plans thetest sequence for the test purpose completed in the previous step, thatis, completes the time planning of the test event represented by it.

S33: developing a test design according to the test concept and testscenario.

Formulation of the test design analyze the previously completed testconcept and the contents of the test scenario to formulate the detailedinformation of the test design, including various aspects of the testfragment, subscription release relationship, data collection plan, andtest sample.

After the test concept and test scenario are completed, the testdesigner can begin to design specific test events. Each test event has acorresponding test fragment and test purpose. The test purpose isderived from the decomposition of the test goal according to the CCI,and it will be used as a reference to decompose the test scenario intothe test fragment. The test fragment is a subset of the test scenario,which will contain all the information for running a test event. A testscenario can be decomposed into multiple test fragments, and each testfragment represents a test event. The design process of the testconsists of the following 4 steps:

S331: formulating a test fragment.

The test fragment needs to be formulated according to the test scenario.The test fragment is a subset of the test scenario. The test fragment isactually a test scenario, composed of participants and the interactionof them. It can be considered that the test fragment is the product ofthe decomposition of the test scenario according to the test purpose.The test purpose corresponds to the key capability issue. The keycapability issue is composed of four key elements, namely standards andconditions, methods and means, task set, and expected effect. Therefore,the test designer can base on the task elements in the key capabilityissue to break down the test scenario. As long as it is judged whichtest resources in the test scenario are related to the task, these testresources are part of the test fragment. In this process, theinteraction of resources will be retained in the test fragment. Fromthis, the test fragment is completed. The relationship among testscenario, test fragment, test goal, and test purpose is shown in FIG.10.

In the subsequent design process, the test designer will add thecorresponding subscription release relationship information, datacollection plan information, and test sample information to the testfragment.

S332: configuring the subscription release relationship.

The subscription release information in the test plan is the basis forthe middleware to transmit data for the components. It describes theobject model transmitted between the subscription party and the releaseparty of the components, that is, determines the data transmissionrelationship among the components during the test operation.

In the design process of the test plan, the subscription releaserelationship is described based on the entity, which is the product ofthe instantiation of the transmission object model of the testparticipants in the plan, including the subscription entity and therelease entity. When the corresponding object model types of thesubscription entity and the release entity are the same, the twoentities can be established as a set of subscription releaserelationship. A subscription release relationship consists ofsubscription participant, release participant, subscription entity,release entity, and subscription party update types. When thesubscription party and the release party are attributes in the objectmodel, the subscription entity and release entity are replaced partly bysubscription entity attributes or events and release entity attributesor events. The subscription participant and the release participantrespectively refer to the names of the participants to which thesubscription party and the release party belong, and they are determinedby the test resource information that has a task interactionrelationship in JOC-T. The instantiation of the test resources has beencompleted when the test scenario is developed, so at that stage, thecorrespondence between the receiver ID and the transmitter ID and namesof the participants to which the subscription party and the releaseparty belong has been completed.

The subscription entity and the release entity refer to the entity namescorresponding to the subscription participant and the releaseparticipant. The subscription entity attribute or event and the releaseentity attribute or event name respectively refer to the correspondingsubscription and release attributes of the entity. An entity is composedof several attributes and several fruit entities. The entity orattribute of the subscription release participant is determined by thetransmission data type in the data interaction matrix. The transmissiondata type is essentially the Chinese interpretation of the entity orattribute. As long as the semantic mapping table between the data typeand the object model type is established, the automatic generation ofthe subscription and release relationship can be realized.

The subscription party update type refers to the method of updating ofthe subscription release relationship data during the test operation,including two modes of direct update and variable value update, whichare determined by the test designer based on the characteristics of thetest.

S333: developing a data collection plan.

The test designer designs the corresponding data collection planaccording to key capability indicator of the CCI corresponding to thecurrent experiment fragment. The data collection plan will determinewhen, how, and by whom collecting data, recording media and formats, andcollecting and receiving standards for data. At this stage, the datacollection requirements should be mapped into a real data collectionplan. A data collection plan contains five elements, namely storagenode, participant, entity name, entity type, and collection method.

The storage node is the node on which the collection software runs andis configured by the experiment designer according to the actualsituation; the participant is the collected participant, which has beendetermined with the completion of the test resource instantiation; theentity name and entity type are the name and type of the data to becollected, and they are determined by the data elements of the keycapability indicators; the collection method is the collection methodused when collecting data, which is divided into event triggering,variable value triggering and timing collection. Event triggering is totrigger the collection when a certain attribute of an entity in theentire test program is equal to a certain value; variable valuetriggering is to trigger the collection when a certain attribute valuechanges; timing collection is to collect once at intervals.

S334: designing a test sample.

The time, cost, manpower and material resources of a test are huge. Inorder to reduce the number of the test required, reduce the cost of thetest, and shorten the cycle of the entire test, it is necessary todesign the correct test sample. For this reason, it is necessary todesign the corresponding test sample for each test fragment. The designprocess and concept are as follows.

S3341: selecting test objects, and setting test attributes andparameters.

S3342: selecting test factors from each test object.

S3343: setting the number of test levels and level values for eachparticipating test factor according to the key capability indicator anddata collection plan in S331.

S3344: selecting a test design method suitable for the selected testfactors.

S3345: generating test samples according to the test design method.

Test object: An object participating in a test, such as a certainparticipant in a chemical experiment, whose attributes can be used astest factors.

Test factor: A parameter that participates in the test and affects theresult of the test. It is a parameter of a certain object, such as thecontent of a certain component in a chemical experiment.

Number of factor levels: The number of variable values of the testfactor. The larger the number of factor levels, the more values that canbe selected for the factor, which will result in more test samples beinggenerated.

Factor level value: the value of the test factor when participating inthe test. A factor can have multiple levels.

Test sample: A permutation and combination of one level value of alltest factors, corresponding to one test.

The various embodiments in this description are described in aprogressive manner. Each embodiment focuses on the differences fromothers, and the same or similar parts among the various embodiments canbe referred to each other. For the device disclosed in the embodiments,since it corresponds to the method disclosed in the embodiments, thedescription is relatively simple, and the relevant part can be referredto the description of the method.

The above description of the disclosed embodiments enables those skilledin the art to implement or use the present invention. Variousmodifications to these embodiments will be obvious to those skilled inthe art, and the general principles defined herein can be implemented inother embodiments without departing from the spirit or scope of thepresent invention. Therefore, the present invention will not be limitedto the embodiments shown in this document, but should conform to thewidest scope consistent with the principles and novel features disclosedin this document.

1. A capability test method based on a joint test support platform, thecapability is defined as an ability to perform tasks to achieve theexpected results through various methods and means under specificstandards and conditions, comprising: S1: describing an initialcapability in a test and determining an evaluation strategy and a jointtask background information of the test according to the capability tobe developed based on the initial capability; S2: generating a logicalshooting range for the joint test support platform according to thejoint task background information; S3: developing a test scenarioaccording to the joint task background information and the logicalshooting range, decomposing the test scenario, and determining a testplan corresponding to the test scenario; S4: performing the testaccording to the test plan; and S5: analyzing and evaluating testresults of the test, and generating one or more joint capabilityevaluation reports for the test.
 2. The capability test method of claim1, wherein the S1 further comprises: S11: formulating a capabilityinformation and a test information; the capability information isinformation describing the capability; the test information isinformation describing the test; the capability is divided into aninitial capability, a developing capability, and an output abilityaccording to the method of the capability in the whole test designstage; the initial capability is defined as a joint capability of a testobject relying on its own equipment/system or a known combination ofsystems or SoS at the initial stage of the test; the developingcapability is defined as a joint capability formed by dividing of thetest object according to task instructions at test planning stage; theoutput capability is defined as an actual output joint capability oftest evaluation analysis stage; S12: developing the joint taskbackground information; retrieving a basic model database of the jointtest environment according to the capability information and testinformation formulated in S11, and selecting the basic model of thejoint test environment suitable for the test; selecting theequipment/system resource model and environmental resource modelparticipating in the test according to the selected basic model of thejoint test environment; adding a task interaction relationship betweenthe equipment/system resource model and the environmental resource modelparticipating in the test, and generating the joint task backgroundinformation; verifying a consistency and completeness of the generatedjoint task background information; S13: developing an evaluationstrategy; collecting the evaluation information; determining a keycapability issue and a key task issue according to the capabilityinformation in S11; and establishing an evaluation strategy framework.3. The capability test method of claim 2, wherein the equipment/systemresource model comprises a first-level platform model and a second-levelplatform model; and the environmental resource model comprises aphysical environment and a human environment.
 4. The capability testmethod of claim 2, wherein the evaluation information in S13 comprises aevaluation subject, an evaluation function, an evaluation area, anevaluation capability, and an evaluation capability level; theevaluation subject is defined as the equipment, personnel orinstitutions for collecting results of test performance andquantitatively evaluating the results of test performance in theevaluation process; the evaluation function is defined as behaviors andactions that the evaluation subject must perform during the evaluationprocess; the evaluation area is defined as a test area for the task tobe evaluated, and the test area is defined by the evaluation subjectbased on the evaluation function; the evaluation capability is definedas a related capability to be evaluated; and the capability level isdefined as the capability evaluation standard for evaluating anddividing the joint capability to be evaluated.
 5. The capability testmethod of claim 2, wherein the key capability issue in S13 is analyticalstatement for evaluating performance of relating capability of a jointmission; basic elements of the key capability issue comprise task set,mission expected effect, test standard, test condition, test method, andtest means; the key task issue is defined as the analytical statement ofdesign task efficiency and mission adaptability; the evaluation strategyframework is divided into independent factor and dependent factoraccording to measured dependence; the independent factor comprises jointmission and task, environmental condition, and system configurationoption; the dependent factor comprises joint mission effectivenessmeasurement, task performance measurement and system propertymeasurement.
 6. The capability test method of claim 2, wherein the S2further comprises: S21: instantiating test resources; selecting asubordinate component of the test resource in the basic model of thejoint test environment to instantiate; S22: adding test node;determining a physical node running by the component according to theinstantiated component, so that the test resource participates in thetest at the correct physical node; S23: generating an intermediateplug-in; and generating a dynamic link library to realize datatransmission among test resources according to the basic model of thejoint test environment corresponding to all test resources in thelogical shooting range.
 7. The capability test method of claim 2,wherein the S3 further comprises: S31: developing a test scenario;wherein the test scenario is a brief description of theaction/interaction relationship between our troops, enemy troops and theenvironment during the test; S32: developing a test concept; wherein thedeveloping a test concept comprises establishing an overall test goal,establishing a test purpose, and determining a test sequence; S33:developing a test design according to the test concept and the testscenario; wherein the test design is a specific plan during testoperation.
 8. The capability test method of claim 7, wherein the S32further comprises: S321: establishing an overall test goal; obtainingthe joint task goal of the test according to the joint task backgroundinformation in S12; associating the available joint task goal with theavailable key capability issue in S13; integrating a stakeholder,capability and joint mission objective into the overall test goal; S322:developing a test purpose; decomposing the global test goal in S321 intothe test purpose according to the key capability issue in S13, and eachtest purpose is based on a specific key capability issue; and S323:determining the test sequence; sorting the developed test purposeaccording to priority and time.
 9. The capability test method of claim7, wherein the S33 further comprises: S331: developing a test fragment;decomposing the test scenario specified in step S31 into a plurality oftest fragments as a test event during test operation by taking the testpurpose in S32 as a reference; S332: configuring a subscription releaserelationship; adding the subscription release relationship for the taskinteraction relationship among test resources, and determining a datatransmission relationship and type among test resources; S333:formulating a data collection plan: determining a key capabilityindicator needed for collecting by the current test fragment accordingto the test fragments in step S331 and the corresponding key capabilityissue, and formulating the corresponding data collection plan; and StepS334: designing a test sample: developing a corresponding test samplefor each test fragment according to the key capability indicator and thedata collection plan in S331.
 10. The capability test method of claim 9,wherein the S334 further comprises: S3341: selecting test objects, andsetting test attributes and parameters; S3342: selecting the test factorfrom each test object; S3343: setting the number of test level and levelvalue for each participating test factor according to the key capabilityindicator and the data collection plan in the S331; S3344: selecting atest design method suitable for the selected test factor, and S3345:generating the test sample according to the test design method.